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Art Unit: 2163 

DETAILED ACTION 

1 . Claims 1-25 are pending in this office action. 

Claim Rejections - 35 USC § 101 
35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

2. Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed subject does 
not comply with the necessary requirements for statutory subject matter. 

With regards to claims 1-11, the claims represent abstract ideas describing the 
steps in a computer program. There is no language in the claims that ties the limitations 
of the claims to being executed on a computer. For example, claim 1 easily reads on a 
person looking through a photo album (image database) for a specific picture 
(representative image). Therefore it is required to include in the preamble of at least 
claims 1 and 5 that the method is computer implemented. Appropriate correction is 
required. 

With regards to claims 12-18, the claims are directed to a computer program per 
se and do not specify that the computer program be embodied on a computer readable 
medium. A computer program product that is not embodied on an acceptable computer 
readable medium is nothing more than an abstract idea. When the computer program 
product is recorded on an acceptable computer readable medium it becomes 
structurally and functionally interrelated to the medium and will be statutory in most 
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cases since use of technology permits the function of the computer program to be 
realized. Appropriate correction is required. 

With regards to claims 19-25, the claims are directed to a computer program gier 
se that is written to a recording medium. A computer program product that is not 
embodied on an acceptable computer readable medium is nothing more than an 
abstract idea. When the computer program product is recorded on an acceptable 
computer readable medium it becomes structurally and functionally interrelated to the 
medium and will be statutory in most cases since use of technology permits the function 
of the computer program to be realized. The applicant must explicitly state in the claim a 
computer readable medium because a recording medium could simply be a piece of 
paper with a program written on it. While a computer readable medium containing a 
computer program is a statutory invention (for the reasons cited above), a computer 
program written on a piece of paper is non-statutory (for reasons cited above). 
Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

3. Claims 1-3 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent Number 6,240,424 issued to Kyoji Hirata (hereinafter "Hirata"). 
Claim 1: 
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Hirata teaches a method of classifying an image, comprising the steps of: 

a) extracting a query image from a plurality of images in an image database 
(Hirata: column 5, lines 58-65); 

b) searching, according to a predetermined similarity level, for a representative 
image resembling the query image in a representative image classification database in 
which groups of images are represented by respective representative images (Hirata: 
column 5, lines 36-40; Note that images are "classified under one primary object". Here the primary 
object is the representative image (See Hirata column 4, lines 66-67). Also note column 4, lines 7-8 and 
column 5, lines 58-65.); 

c) registering the query image as a new representative image in the 
representative image classification database when no resembling representative image 
is found as a result of the search according to the predetermined similarity level (Hirata: 
column 4, lines 1-14; This reference shows the steps for creating a new representative image. This would 
be done when an image does not fit into any category already determined.); and 

d) adding the query image into a group represented by the resembling 
representative image found as a result of the search according to the predetermined 
similarity level (Hirata column 4, lines 9-14 and column 7, lines 7-10). 

Claim 2: 

Hirata teaches all the elements of claim 1, as noted above, and Hirata further 
teaches wherein the images in the image database are obtainable by referring to the 
respective representative images in accordance with the predetermined similarity level 

(Hirata: column 5, line 58-column 6, line 1 and Fig. 5; Note that the primary object is the representative 
image as shown at column 4, lines 66-67.). 



Application/Control Number: 10/619,497 Page 5 

Art Unit: 2163 

Claim 3: 

Hirata teaches all the elements of claim 1, as noted above, and Hirata comprising 
a step of forming the groups into a hierarchical structure (Hirata: Figs. 2A, 2B and also 
column 5, lines 38-40; Note in column 5, lines 38-40 states "classified under one primary object". This 
further shows the hierarchical structure of images under a representative image (here the primary 
object).), wherein the forming step further includes the steps of: 

a) extracting a further query image from the representative images in the 
representative images Classification database (Hirata: column 5, lines 36-40; Note that images 
are "classified under one primary object". Here the primary object is the representative image (See Hirata 
column 4, lines 66-67). Also note column 4, lines 7-8 and column 5, lines 58-65.); 

b) searching, according to a further predetermined similarity level, for a further 
representative image resembling the further query image in a further representative 
image classification database in which groups of images are represented by respective 
further representative images (Hirata: column 5, lines 36-40; Note that images are "classified under 
one primary object". Here the primary object is the representative image (See Hirata column 4, lines 66- 
67). Also note column 4, lines 7-8 and column 5, lines 58-65.); 

c) registering the further query image as a new further representative image in 
the further representative image classification database when no resembling further 
representative image is found as a result of the search according to the further 
predetermined Similarity level (Hirata: column 4, lines 1 -14; This reference shows the steps for 
creating a new representative image. This would be done when an image does not fit into any category 
already determined.); and 
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adding the further query image into a group represented by the resembling 
further representative image found as a result of the search according to the further 

predetermined Similarity level (Hirata column 4, lines 9-14 and column 7, lines 7-10). 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hirata in 
further view of U.S. Patent Application Publication Number 2003/001 1683 issued to 
Fumitomo Yamasaki et al. (hereinafter "Yamasaki"). 
Claim 4: 

Hirata discloses all the elements of claim 3, as noted above, but does not 
explicitly disclose wherein the hierarchical structure is formed as layers of a directory of 
a file system for managing the images in the image database. 

However, Yamasaki discloses wherein the hierarchical structure is formed as 
layers of a directory of a file system for managing the images in the image database 

(Yamasaki: paragraphs [0087], [0089], and Fig. 9). It would have been ObviOUS to One of 

ordinary skill in the art at the time the invention was made to modify the teachings of 
Hirata with the teachings of Yamasaki noted above forming a directory structure of 
images (Yamasaki: paragraph [0089]). The skilled artisan would have been motivated to 
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improve the teachings of Hirata per the above such that using the hierarchical structure, 
the user can readily sort out image data (Yamasaki: paragraph [0091], lines 1-5). 

5. Claims 5-7, 12-14, and 19-21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over NPL article titled "Recursive Space Decompositions in Force- 
Directed Graph Drawing Algorithms" written by K.J. Pulo (hereinafter "Pulo") if further 
view of U.S. Patent Application 2003/0198384 issued to Michael Vrhel (hereinafter 
"Vrhel"). 
Claim 5: 

Pulo discloses an image feature space display method comprising the steps of: 

a) determining k representative points (k being an integer which is more than 1) 
in a feature space in response to a distance between points in the feature space and 
representative points representative of a plurality of feature spaces surrounding the 
feature space (Pulo: Section 3.2 Finding The Characteristic Points, lines 39-40; Note that the k-means 
algorithm performs the all the same functionality described here. Section 2.2, lines 8-15 further describes 
the grouping of objects with respect to proximity and spatial location (distance).); 

b) obtaining k sub-feature spaces by evenly allocating the points in the feature 
space into k representative points (Pulo: Section 3.2 Finding The Characteristic Points, lines 41- 
43); 

c) dividing a display space into sub-display regions of k segments, the display 
space being divided in a manner so that the sub-feature spaces correspond to the sub- 
display regions (Pulo: Section 2.2 Recursive Space Decompositions (RSDs), lines 1-5; Note that this 
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describes the function of an RSD. The k-means algorithm is a type of RSD (Section 2.2 Recursive Space 
Decompositions (RSDs), lines 41-45).)! 

d) repeating the steps a) through c) until the sub-feature spaces and the sub- 
display regions are divided into minimum units, respectively (Pulo: Section 2.2 Recursive 

Space Decompositions (RSDs), lines 2-3 and Section 3.2 Finding The Characteristic Points, lines 43-45; 
These two citations display a clear reference to the recursive nature of the k-means algorithm which 
results in the repeating of steps a, b, and c); and 

Pulo discloses the k-means algorithm for carrying out the previous limitations 
cited above, but Pulo does not explicitly disclose applying the k-means function for 
segmenting images. However, Vrhel discloses applying the k-means function to images 
for the purpose of segmenting the image (Vrhel: paragraph [0017], lines 6-9). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify the teachings of Pulo with the teachings of Vrhel noted above. The skilled 
artisan would have been motivated to apply the teachings of Pulo per the above in order 
to classify individual pixels from an image into particular groups (Vrhel: paragraph [001 0], 

lines 1-4). 

Pulo does not explicitly disclose e) arranging each image included in a minimum 
unit of a sub-feature space to a corresponding one of the minimum units of the sub- 
display regions. However, Vrhel discloses e) arranging each image included in a 
minimum unit of a sub-feature space to a corresponding one of the minimum units of the 
sub-display regions (Vrhel: paragraph [0018], lines 17-21). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify the teachings of 
Pulo with the teachings of Vrhel noted above for the purpose of having units of a feature 
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space corresponding to units of a display (image) (Vrhel: paragraph [0018], lines 17-21). The 
skilled artisan would have been motivated to improve the teachings of Pulo per the 
above such that each feature identifies a particular portion of the display (image) (Vrhel: 
paragraph [0003], lines 8-11). 
Claim 6: 

The combination of Pulo and Vrhel discloses all the elements of claim 5, as noted 
above, and Pulo further discloses wherein the display space is two dimensional (Pulo: 
Section 2.2 Recursive Space Decompositions (RSDs), lines 13-15), wherein the feature space and 
the display space are divided into four sub-feature spaces and four sub-display regions 
in a grid manner (Pulo: Section 2.2, lines 46-55 and Fig. 2), respectively, wherein the 
representative points are disposed proximally with respect to two feature spaces which 
are arranged adjacent to each other and tangent to the sub-feature spaces, and thus 
disposed distally with respect to two other feature spaces which are arranged adjacent 
to each other but not tangent to the sub-feature spaces (Pulo: Fig. 2). 
Claim 7: 

The combination of Pulo and Vrhel discloses all the elements of claim 5, wherein 
the display space is three-dimensional (Pulo: Section 3.1 Description, lines 1-2; This reference 
suggests a handling a variable dimension space (d-dimensional). So three dimensional must be one of the 
cases considered.), wherein the feature space and the display space are divided into eight 
sub-feature spaces and eight display regions in a grid manner (Pulo: Section 2.2, lines 16- 
30; The specific example chosen by Pulo is one that divides regions by 4. This is further seen in Fig. 2. 
However, in Section 2.2, lines 20-22, it is explicitly stated that irregular RSDs (like the k-means 
algorithm) "may divide space into arbitrarily sized and shaped regions at each level." This surely covers 
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all types of sub-divisions, including where the feature space and display regions are divided by 8.), 
respectively, wherein the representative points are disposed proximally with respect to 
three feature spaces which are arranged adjacent to each other and tangent to the sub- 
feature spaces, and thus disposed distally with respect to three other feature spaces 
which are arranged adjacent to each other but not tangent to the sub-feature spaces 

(Pulo: This can be seen with respect to two dimensions in Fig. 2. While a diagram is not given for an 
example of 3-dimensions, the references cited above in the rejection of this claim state that it would be 
possible to have a 3-dimensional space (d-dimensional) and divide by the sub-feature space and display 
region by 8 ("arbitrarily sized and shaped regions").). 
Claim 12: 

Claim 12 is a computer program product claim corresponding to method claim 5 
and is rejected under the same reasons set forth in the rejection of claim 5. 
Claim 13: 

Claim 13 is a computer program product claim corresponding to method claim 6 
and is rejected under the same reasons set forth in the rejection of claim 6. 
Claim 14: 

Claim 14 is a computer program product claim corresponding to method claim 7 
and is rejected under the same reasons as set forth in the rejection of claim 7. 

Claim 19: 

Claim 19 is a computer program product claim corresponding to method claim 5 
and is rejected under the same reasons set forth in the rejection of claim 5. 
Claim 20: 
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Claim 20 is a computer program product claim corresponding to method claim 6 
and is rejected under the same reasons set forth in the rejection of claim 6. 
Claim 21: 

Claim 21 is a computer program product claim corresponding to method claim 7 
and is rejected under the same reasons set forth in the rejection of claim 7. 
6. Claims 8-9, 15-16, and 22-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pulo in further view of Vrhel and further in view of Hirata. 
Claim 8: 

The combination of Pulo and Vrhel discloses all the elements of claim 5, as noted 
above, but the combination does not explicitly disclose wherein the points in the feature 
space represent images in a representative image classification database which are 
subject to the steps of: 

a) extracting a query image from a plurality of images in an image database; 

b) searching, according to a predetermined similarity level, for a representative 
image resembling the query image in the representative image classification database 
in which groups of images are represented by respective representative images; 

c) registering the query image as a new representative image in the 
representative image classification database when no resembling representative image 
is found as a result of the search according to the predetermined similarity level; and 

d) adding the query image into a group represented by the resembling 
representative image found as a result of the search according similarity level the 
predetermined. 
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However, Hirata discloses wherein the points in the feature space represent 
images in a representative image classification database which are subject to the steps 
of: 

a) extracting a query image from a plurality of images in an image database 
(Hirata: column 5, lines 58-65); 

b) searching, according to a predetermined similarity level, for a representative 
image resembling the query image in the representative image classification database 
in which groups of images are represented by respective representative images (Hirata: 
column 5, lines 36-40; Note that images are "classified under one primary object". Here the primary 
object is the representative image (See Hirata column 4, lines 66-67). Also note column 4, lines 7-8 and 
column 5, lines 58-65. ); 

c) registering the query image as a new representative image in the 
representative image classification database when no resembling representative image 
is found as a result of the search according to the predetermined similarity level (Hirata: 
column 4, lines 1-14; This reference shows the steps for creating a new representative image. This would 
be done when an image does not fit into any category already determined.); and 

d) adding the query image into a group represented by the resembling 
representative image found as a result of the search according similarity level the 

predetermined (Hirata column 4, lines 9-14 and column 7, lines 7-10). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the previously mentioned combination with the teachings 
of Hirata noted above for the purpose of classifying and querying a database of images 
(Hirata: Abstract). The skilled artisan would have been motivated to improve the 
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previously mentioned combination per the above such images could be classified to a 
group based on the similarity to a representative image of the group (Hirata: column 4, lines 
1-14). 
Claim 9: 

The combination of Pulo, Vrhel, and Hirata discloses all the elements of claim 8, 
as noted above, and Hirata further discloses comprising a step of forming the groups 
into a hierarchical structure, wherein the forming step further includes the steps of: 

a) extracting a further query image from the representative images in the 
representative image classification database (Hirata: column 5, lines 58-65); 

b) searching, according a further predetermined similarity level, for a further 
representative image resembling further query image in a further representative image 
classification database in which groups of images are represented by respective further 

representative images (Hirata: column 5, lines 36-40; Note that images are "classified under one 
primary object". Here the primary object is the representative image (See Hirata column 4, lines 66-67). 
Also note column 4, lines 7-8 and column 5, lines 58-65.); 

c) registering the further query image as a new further representative image in 
the further representative image classification database when no resembling further 
representative image is found as result of the search according to the further 
predetermined similarity level (Hirata: column 4, lines 1-14; This reference shows the steps for 
creating a new representative image. This would be done when an image does not fit into any category 
already determined.); and 
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d) adding the further query image into a group represented by the resembling 
further representative image found as a result of the search according to the further 
predetermined similarity level (Hirata column 4, lines 9-14 and column 7, lines 7-10). 
Claim 15: 

Claim 15 is a computer program product claim corresponding to method claim 8 
and is rejected under the same reasons set forth in the rejection of claim 8. 
Claim 16: 

Claim 16 is a computer program product claim corresponding to method claim 9 
and is rejected under the same reasons set forth in the rejection of claim 9. 
Claim 22: 

Claim 22 is a computer program product claim corresponding to method claim 8 
and is rejected under the same reasons set forth in the rejection of claim 8. 
Claim 23: 

Claim 23 is a computer program product claim corresponding to method claim 9 
and is rejected under the same reasons set forth in the rejection of claim 9. 
7. Claims 10-11, 17-18, and 24-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pulo in further view of Vrhel and further in view of U.S. Patent 
Application Publication Number 2003/0059121 issued to Andreas E. Savakis et al. 
(hereinafter "Savakis"). 
Claim 10: 

Pulo discloses an image feature space display method comprising the steps of: 



Application/Control Number: 10/619,497 Page 15 

Art Unit: 2163 

a) dividing a feature space into three sub-feature spaces, the three sub-feature 
spaces being composed two sub-feature spaces disposed within a prescribed radius 
with respect to two reference points in the feature space, and another sub-feature space 
Other than the two SUb-feature spaces (Pulo: Section 3.2 Finding The Characteristic Points, lines 
39-40; Note that the k-means algorithm performs the all the same functionality described here. Section 
2.2, lines 8-15 further describes the grouping of objects with respect to proximity and spatial location 
(distance). Further note that Pulo discloses dividing into a k (variable) amount of subsections.); 

b) dividing a display space into sub-display regions of three segments, the 
display space being divided a same manner as the feature space so that the sub- 
feature spaces correspond to the sub-display regions (Pulo: Section 2.2 Recursive Space 
Decompositions (RSDs), lines 1-5; Note that this describes the function of an RSD. The k-means 
algorithm is a type of RSD (Section 2.2 Recursive Space Decompositions (RSDs), lines 41-45).); 

c) repeating the steps a) and b) the sub-feature spaces and the sub-display 
regions are divided into minimum units, respectively (Pulo: Section 2.2 Recursive Space 
Decompositions (RSDs), lines 2-3 and Section 3.2 Finding The Characteristic Points, lines 43-45; These 
two citations display a clear reference to the recursive nature of the k-means algorithm which results in 
the repeating of steps a, b, and c.)i and 

Pulo discloses the k-means algorithm for carrying out the previous limitations 
cited above, but Pulo does not explicitly disclose applying the k-means function for 
segmenting images. However, Vrhel discloses applying the k-means function to images 
for the purpose of segmenting the image (Vrhel: paragraph [0017], lines 6-9). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify the teachings of Pulo with the teachings of Vrhel noted above. The skilled 
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artisan would have been motivated to apply the teachings of Pulo per the above in order 
to classify individual pixels from an image into particular groups (Vrhel: paragraph [0010], 

lines 1-4). 

Pulo does not explicitly disclose e) arranging each image included in a minimum 
unit of a sub-feature space to a corresponding one of the minimum units of the sub- 
display regions. However, Vrhel discloses e) arranging each image included in a 
minimum unit of a sub-feature space to a corresponding one of the minimum units of the 
sub-display regions (Vrhel: paragraph [0018], lines 17-21). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify the teachings of 
Pulo with the teachings of Vrhel noted above for the purpose of having units of a feature 
space corresponding to units of a display (image) (Vrhel: paragraph [0018], lines 17-21). The 
skilled artisan would have been motivated to improve the teachings of Pulo per the 
above such that each feature identifies a particular portion of the display (image) (Vrhel: 

paragraph [0003], lines 8-11). _ 

The combination of Pulo and Vrhel does not explicitly disclose dividing the 
feature space and the display space into specifically three subsections. However, 
Savakis discloses using the k-means function to divide subject matter into three 
subsections (Savakis: paragraph [0082], lines 2-4). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify the previously 
mentioned combination with the teachings of Savakis noted above. The skilled artisan 
would have been motivated to improve the previously mentioned combination because 
Pulo specifically suggests that the subdivisions produced by the k-means algorithm (an 
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irregular RSD) may divided subject matter into any arbitrary size and shape (Pulo: Section 
2.2, lines 20-22; Therefore a larger shape results in less subdivisions and larger shape results in more 
subdivisions. This leaves selecting 3 subdivisions as a design choice.). 
Claim 11: 

The combination of Pulo, Vrhel, and Savakis discloses all the elements of claim 
10, as noted above, and Pulo further discloses wherein the reference points are 
selected from points disposed nearest to representative points included in the two sub- 
feature spaces (Pulo: Section 3.2, lines 39-43 and Section 2.2, lines 8-15). 
Claim 17: 

Claim 17 is a computer program product claim corresponding to method claim 10 
and is rejected under the same reasons set forth in the rejection of claim 10. 
Claim 18: 

Claim 18 is a computer program product claim corresponding to method claim 1 1 
and is rejected under the same reasons set forth in the rejection of claim 1 1 . 
Claim 24: 

Claim 24 is a computer program product claim corresponding to method claim 10 
and is rejected under the same reasons set forth in the rejection of claim 10. 
Claim 25: 

Claim 25 is a computer program product claim corresponding to method claim 1 1 
and is rejected under the same reasons set forth in the rejection of claim 1 1 . 
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0788. The examiner can normally be reached on Monday - Friday, 9:00 am - 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on (571) 272-4023. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




